VidArch: Preserving Meaning of Digital Video over Time through Creating and Capture of Contextual Documentation

The NSF/LC-funded VidArch project is moving beyond a focus on the preservation of data and isolated information objects to the preservation of persistent context that will make today’s videos not only accessible, but also understandable far into the future. This project builds from the Reference Model for an Open Archival Information System (OAIS) but also uses the archivist’s traditional descriptive tool, the finding aid. The VidArch project is blending the conceptualizing power of the finding aid with the complexity and information-rich nature of video to create the video-, and more generally, multimedia-enhanced finding aid. This paper reports on our progress toward developing a framework of elements to be documented within video collections (related actors, events, objects, places and times), and a cost-effective means to locate, authenticate, select, and capture these materials within preservation environments. In this process we seek to identify those elements that are best documented today and secure contextualizing materials for them while noting other elements that will be more deeply supported by materials available in the future. Digital Preservation and Video Digital preservation is one of the grand challenges of the early 21st Century. [1-4] If future generations are to have a record of today’s scholarly, scientific, and cultural achievements, we must learn how to preserve digital assets, whether they are textual documents, research data, or complex multimedia objects. To this time, much of the discussion of, and approaches to, digital preservation have focused on 1) technical issues associated with preserving the bits; 2) social engineering as seen in records management compliance and identification of incentives to preserve; and 3) the creation, capture, and acquisition of metadata to support future discovery, administration, integration, and physical preservation of digital files. Video content offers special challenges for preservation just as it offers special challenges for digitization and retrieval. Most importantly, video is a temporal medium with multiple information representation channels. Multiple visual (primary visual activity, superimpositions of text or graphics) and audio (including the possibility of human speech in multiple languages, music, natural and artificial sound effects) channels may also be augmented by human-readable or machine readable content (e.g., closed 1 This work was supported by a grant from the National Science Foundation #IIS 0455970 captioning or feature vectors, respectively). A video object’s meaning is greater than the sum of its parts, and preservation must attend to the whole as well as the parts. Much of the preservation research to date has centered on text and still images, but moving and multi image materials have received significant attention as well. [5-8] This project builds on earlier work with digital video files and their surrogates seeking ways in which to preserve a video work’s context and highlighting its essence, thus making it more understandable and accessible to future generations. [9-10] For digital objects, preservation and access are integrally intertwined. Conway [11] argues that unlike physical materials, for which active use can make preservation more difficult, only those digital objects that remain accessible and that are used will be preserved over generations of costly changes to software and hardware. Here “preservation is a reality and not merely a metaphor for or symbol of access” and accessibility and use promote longevity. Lavoie and Dempsey [12] illustrate the range of ways that preservation may be viewed and note that “mechanisms to ensure long-term persistence should operate harmoniously with mechanisms supporting dissemination and use.” Thus, long-term provision of contextualized access that makes digital objects understandable over time is essential to longterm preservation. It is our contention that metadata is crucial for access and substantial context is crucial for understanding over time. We, therefore, are focusing in this research project on developing a preservation framework for digital video context. There has been remarkable progress made in creating metadata schemes and standards for digital content (e.g., DC, METS, NLNZ) [13-15] and recent work to develop a strong foundation for digital preservation metadata (PREMIS) [16]. However, much remains to be learned about long-term access considerations and how they should be encoded into information systems and metadata specifications. Although metadata will ultimately specify the infrastructural aspects of context, we have little evidence on the kind or extent of contextual information necessary and sufficient to preserve digital entities (object-specific annotations let alone general ‘state of the world’ at time of creation) in an accessible and comprehensible manner. This is a general problem for digital preservation, but we posit that it is especially problematic for digital video content. Creation and capture of context is central to archival theory and practice. Documentation of context can make materials useful and comprehendible in the future by individuals who were not involved in the activities that are documented by the materials. This is the hallmark of a well-written archival finding aid, a device that has proven useful, if expensive, for describing and providing access to primarily text-based collections. Encoded Archival Description (EAD) [17] is a standard designed to make finding aids more accessible and interoperable. As finding aids themselves become digital objects, they can take advantage of the digital medium to include added value elements such as multiple media, hyperlinks, and interactive behaviors that make them more representative and more usable. This project seeks to develop approaches for addressing the capture of existing and creation of new metadata surrogates to ensure that collections of digital video will be meaningful and accessible over the long-term. These surrogates will include text representations, but most importantly and uniquely, will employ elements from video to describe video. More importantly, we aim to define a digital video preservation framework focused on context that can be used by archivists to make preservation decisions and guide the development of description and access systems. The framework will include cost considerations from several angles including the degree to which elements may be automated. We will demonstrate this framework by applying it to two important digital video collections held within the Open Video repository at the University of North Carolina at Chapel Hill. Open Video Open Video (OV) (http://www.open-video.org), housed within the School of Information and Library Science at the University of North Carolina at Chapel Hill collects and makes available digitized video content for the digital video, multimedia retrieval, digital library, and other research communities (see Figure 1). Figure 1. Open Video Website Significantly, Open Video began as a project to provide researchers with access to video to facilitate the study of problems, such as the development of algorithms for automatic segmentation, summarization, and creation of surrogates that describe video content; and the creation and evaluation of interfaces that display result sets from multimedia queries. Begun in 1998 with a framework and the digitization of approximately 195 video segments, Open Video has been very successful in collecting a wide range of video content and now finds itself a repository with preservation as well as access needs. This situation, the diverse nature of the materials, and the complex workflow process that results in multiple surrogates such as fast forwards and storyboards, led the DigArch team to use content from OV as initial case studies. Specifically, we are using video from the U.S. National Aeronautics and Space Administration (NASA) Langley Center for Distance Learning and the Association for Computing Machinery (ACM) collections within OV to explore what types of contextual material digital video curators might preserve along with actual videos so as to facilitate enhanced understanding for future generations of users. We are also developing a decision framework to assist curators in the selection and extent of such contextualizing information using the Open Video environment. The NASA Collection In the Fall of 2003 OV began working with NASA Langley’s Center for Distance Learning to provide digital video file versions of four educational programs. The programs are designed to (1) enhance and enrich science, technology, engineering, and mathematics (STEM)-related teaching and learning; (2) advance the theory and practice of teaching mathematics, science, and technology; (3) inspire and encourage students (especially women and minorities) to pursue STEM-related courses and careers; and (4) increase (adult) scientific and technological literacy. Through this collaboration, teachers, students, and parents can now access and download the complete set of programs from an Open Video server. The programs -NASA Kids Science Network,TM the NASA Sci FilesTM, NASA CONNECTTM and NASA's Destination TomorrowTM -air on PBS, cable, and ITV and are used nationally by about 500,000 educators, representing about 10.6 million students. At present, there are 55 segments from NASA Kids Science News NetworkTM, 188 segments from 17 NASA SCI FilesTM, 222 segments from 35 NASA CONNECTTM, and 75 segments from 14 NASA's Destination TomorrowTM programs. Each segment and program is available in MPEG-1, MPEG-2, MPEG-4, QuickTime, and Real formats to allow users with different connectivity and software to use the materials. Thus, there are 2,700 video files (5 versions of each segment), 540 metadata records in a MySQL database, and more than 11,000 keyframe files (GIFs) used in storyboards as well as 540 QuickTime fast forwards in the collection. Figure 2 illustrates what an OV user would see when coming to The Red Planet video from NASA. Many of the same videos are also available at nasa.ibiblio.org. A collaboration of the Center for the Public Domain and The University of North Carolina Chapel Hill, ibiblio is “a conservancy of freely available information, including software, music, literature, art, history, science, politics, and cultural studies” (www.ibiblio.org/about.html). The video segments come from NASA and pass through the OV workflow with copies and various surrogates placed in both locations. This is more fully explained in the VidArch section below. See Figure 3 for nasa.ibiblio.org’s presentation of the NASA videos. Here the audience is clearly K-16 educators, families, and students while the OV targeted audience also aims to serve video and information science researchers as well as the public. This is an excellent example of how the access systems associated with the same digital objects can be provided very different look and feel. Such customizable dissemination information packages (DIPs) are an essential element of the OAIS Reference Model. Figure 2. :The Red Planet” Video from Open Video Figure 3. nasa.ibiblio.org. The ACM Collection The second collection we are using to evaluate the digital video context preservation framework consists of videos presented at research conferences sponsored by the Association for Computing Machinery (ACM). One conference in particular, the Annual conference of the ACM Special Interest Group in Computer Human Interaction (SIGCHI) has accepted juried videos each year since 1983. From 1983 through 2002 these videos were distributed on NTSC videotape to conference attendees and in more recent years, they were distributed on DVDs. Researchers in human-computer interaction produce these videos at their university or corporate laboratories and submit them for peer review. Each year, a small number (typically around a dozen videos ranging in length from a few minutes to 15 minutes or more) are accepted. Open Video has collected digital versions of all the SIGCHI videos from 1983 through 2003 (427 videos). The process by which these videos were digitized varied. More importantly, these videos were produced by a variety of people around the world with a variety of audio-visual skill and resources (e.g., some were professionally produced by large corporate labs and others shot by hand by university students). Thus, the original visual and audio quality varies enormously. For all of these videos, Open Video includes metadata by using the keywords and other information authors provided to the ACM conference. Thus for the SIGCHI videos, OV has 1,708 video files (4 versions of each video), 427 metadata records in the MySQL database, approximately 15,000 keyframe files (GIFs) as well as 427 QuickTime fast forward files. Digital Archiving and the OAIS Reference Model This project builds from the Reference Model for an Open Archival Information System (OAIS), now an ISO standard. [18] The OAIS is a high-level conceptual model, intended to serve as the basis for more specific standards and approaches as it articulates “the functionality and components of any system responsible for preserving any type of information....”[19] It indicates the importance of contextual information, but it does not describe how to specify the contextual information. One of the goals of the VidArch project is to further flesh out the notion of context, particularly as it pertains to the preservation of digital video collections within OAIS repositories. The OAIS Reference Model distinguishes between three types of information packages. The Submission Information Package (SIP) is “information delivered by the producer of the archival content to the OAIS repository for use in the construction of one or more Archival Information Packages (AIP).” An AIP is “the content information that is a target of preservation and the associated Preservation Description Information (PDI), which is preserved within the repository along with the digital object(s).” Finally, the Dissemination Information Package (DIP) is “information taken from one or more AIPs, and packaged with the target content for the user of the repository.” [20] In this project we are investigating reliable and cost-effective approaches for curators of video collections to turn sets of discrete video files into collections of context-rich Archival Information Packages (AIPs). The VidArch Approach The Video Enhanced Finding Aid The NSF/LC-funded VidArch project is moving beyond the focus on the preservation of data and isolated information objects to the preservation of persistent context that will make today’s videos not only accessible, but also understandable far into the future. VidArch researchers are blending the conceptualizing power of the finding aid with the complexity and information-rich nature of video to create the video-, and more generally, multimedia-enhanced finding aid. Our first step has been to create conventional Encoded Archival Description (EAD) finding aids for the NASA and ACM collections (see Figure 4). These finding aids contain several levels of hierarchy from the entire collections through series and subseries of videos down to listing of individual videos and segments broken out of each. These tools provide both a structured hierarchical view of the collections beginning with broad collective descriptions and the opportunity to provide varying degrees of granularity at any given level within the hierarchy. Figure 4. Finding Aid for the NASA K-16 Science Traditionally, archivists have used finding aids, first in paper format and more recently encoded in XML and mounted on repository websites, to provide users with contextual information to understand the documents within a collection. This contextual information most frequently relates to the entities involved in the creation of the records -individuals, organizations and functions and the nature of the records themselves. Organization of records by provenance, document type, and often chronological order -and the capture of this arrangement within the finding aid – can assist users who may be many generations removed from the creation of the original records. Rather than simply being an aggregation of existing metadata, the creation of findings aids often involves substantial intellectual work and professional judgment on the part of archivists. We, therefore, contend that finding aids should be considered not just access devices but also digital objects that themselves should be submitted to and ingested into the OAIS along with the target videos. With the EAD finding aids as a framework, we are presently identifying a typology of elements (related actors, events, objects, locations, times), to be documented within video collections by examining the contents of video segments. We are starting with videos from the NASA collection, such as The Red Planet and seeking approaches for determining how best to locate, authenticate, select, and embed these materials within or attach them to the preservation package. In this process we seek to identify those elements that are best documented today and secure contextualizing materials for them while noting other elements that will be more deeply supported by materials available in the future. Our immediate goal is to produce and test the feasibility of a multimedia finding aid that documents video with not only text but also video and other multimedia objects. An example from a NASA video about hurricanes is indicative of these efforts. Weather balloons are mentioned in this program as a tool that helped meteorologists forecast and assess hurricanes and it is noted that they are being replaced with data from satellites and other sensor systems. Given the progress of weather forecasting tools and technologies it is very likely that in 50 years few individuals viewing this educational program would have any idea how today’s weather balloons are used or what they looked like, even though the hurricane program gives a fairly clear discussion of the device’s purpose and the type of data it collected. This seems an excellent element to document with images and not merely descriptive text. Also, it would seem prudent to document this object now, while images and video of weather balloons are easy to locate and preserve along with the target hurricane video. Contextual Elements from the Open Video Workflow Process Simultaneously, we are developing a framework of contextual elements to be documented from creator-provided information at the time of ingest. Our work begins with a general model of the life cycle of video objects. Figure 5. Life Cycle of Digital Videos from NASA in Open Video Figure 5 represents a series of stages in the life cycle of digital videos from NASA that are available from Open Video and nasa.ibiblio. The solid arrows in the figure show current information flows, while the dashed arrows indicate potential for capturing further contextual information. The first point in the life cycle is the functional provenance within NASA. This includes the mission, goals, objectives and programs within NASA that generated the videos. The second stage involves the activities undertaken by both NASA and its contractors to produce the videos. Next is NASA distribution of the video, which includes television broadcasts, posting to NASA web sites, and NASA direct dissemination to classrooms. Periodically, NASA sends to the Open Video team a physical medium -either video tape or DVD -through postal mail and a SMIL file (including transcript and timestamp data) separately as an email attachment. The Open Video team views the analog segments contained on the physical media and reviews associated NASA documentation to generate metadata elements, which they use to populate database records associated with each video. Then the team uses Final Cut Pro to generate a large master digital video file in .mov format. Again using Final Cut Pro, they divide each video into a set of much smaller segments, based on what they judge to be useful cut points. For each segment, the Open Video team uses a product called MediaCleaner to derive copies in five different and smaller formats: MPEG-1, MPEG-2, MPEG-4, QuickTime and Real. They also create a .RAM file for the audio that accompanies the Real version. Next they create storyboards that contain representative keyframes. They use a program called MERIT to generate a set of candidate keyframes for each video segment. They then manually choose a set of approximately 18-36 keyframes that they judge to be most representative of the segment and discard the rest. While making these choices they also categorize keyframes that contain faces and text superimpositions. They also choose one of the keyframes to serve as the single “posterframe” to be a useful representation of the entire segment (see the larger image of the Martian surface in Figure 6 for The Red Planet). Next, they also generate “fast forwards,” which are surrogates that present every nth frame, allowing users to quickly get a sense of what the segment contains. Seven-second excerpts of the video are also extracted to serve as a third kind of non-textual surrogate. The resulting set of video segments, surrogates, and other metadata elements are served from the Open Video site. The video and metadata are served from the nasa.ibiblio.org site. The experience of navigating the two sites is quite different, because they are designed for different use scenarios. Figure 6. Posterframe for “The Red Planet”